Evaluating Reliability of Motion Features in Surveillance Videos

Although a tremendous effort has been made to perform a reliable analysis of images and videos in the past fifty years, the reality is that one cannot rely 100% on the analysis results. The only exception is applications in controlled environments as dealt in machine vision, where closed world assumptions apply. However, in general, one has to deal with an open world, which means that content of images may significantly change, and it seems impossible to predict all possible changes. For example, in the context of surveillance videos, the light conditions may suddenly fluctuate in parts of images only, video compression or transmission artifacts may occur, a wind may cause a stationary camera to tremble, and so on. The problem is that video analysis has to be performed in order to detect content changes, but such analysis may be unreliable due to the changes, and thus fail to detect the changes and lead to “vicious cycle”. The solution pursuit in this paper is to monitor the reliability of the computed features by analyzing their general properties. We consider statistical properties of feature value distributions as well as temporal properties. Our main strategy is to estimate the feature properties when the features are reliable computed, so that any set of features that does not have these properties is detected as being unreliable. This way we do not perform any direct content analysis, but instead perform analysis of feature properties related to their reliability. The main effort in video analysis nowadays is still in making the feature computation more reliable. Our claim is that we need to accept the fact that the computed features will never be 100% reliable, and focus our attention on computing reliability measures. This way system decisions will only be made when features are sufficiently reliable. This means for an intelligent system for video analysis that in addition to feature computation, it should perform instantaneous evaluation of their reliability, and adapt its behavior in accordance to the reliability. For example, if the goal of a system is to monitor motion activity, and to signal an alarm if the activity is high, the system is allowed to make reliable decisions only if there exist a subset of the computed motion activity features that is sufficiently reliable. The monitoring of features reliability and adjusting the system behavior accordingly, seems to be the only way to deploy autonomous video surveillance systems.